Tried to run the tests locally but ran into:

Caused by:
  failed to create directory `/usr/share/testsuite/rust/cargo_build/target`

Caused by:
  Cross-device link (os error 18)
# ...
    [2] autobuild(dir::AbstractString, src_name::AbstractString, src_version::VersionNumber, sources::Vector{<:AbstractSource}, script::AbstractString, platforms::Vector, products::Vector{<:Product}, dependencies::Vector{<:AbstractDependency}; verbose::Bool, debug::Bool, skip_audit::Bool, ignore_audit_errors::Bool, autofix::Bool, code_dir::Union{Nothing, String}, require_license::Bool, dont_dlopen::Bool, kwargs::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
      @ BinaryBuilder ~/repos/BinaryBuilder.jl/src/AutoBuild.jl:932
    [3] (::var"#25#27"{Platform, Vector{Platform}})(build_path::String)
      @ Main ~/repos/BinaryBuilder.jl/test/building.jl:132

JuliaPackaging/BinaryBuilderBase.jl#395

Where do the hello_world_... binaries come from?

You mean the https://github.com/JuliaBinaryWrappers/HelloWorldC_jll.jl package? That (and friend) are normal packages in Yggdrasil.

That provides hello_world; where do these come from?

Because those fail during testing:

[ Info: ["hello_world_ocaml"] does not exist, reporting unsatisfied
┌ Error: Built testsuite but hello_world_ocaml still unsatisfied:
└ @ BinaryBuilder /data/staticfloat/bb_azp/rootfs/agent_3/_work/1/s/src/AutoBuild.jl:973
##[error]Built testsuite but hello_world_ocaml still unsatisfied:

Ah, the testsuite is in https://github.com/JuliaPackaging/Yggdrasil/tree/1faf9cffaaeff93f8e9c0ae48b2e75ec19545147/0_RootFS/Rootfs/bundled/testsuite, needs building a new RootFS as well...

This is an area where BB2 would hopefully be better and we can dynamically generate all these extra files on the file in one package, instead of having to build a new layer for each thing....

So that just needs to contain the source files of a hello_world-generating binary? Or would it be fine to add some code generating it to the build script; it's a simple Hello World print after all?

So that just needs to contain the source files of a hello_world-generating binary?

Yes, it pretty much needs a Makefile to generate the executable and that's it.

Or would it be fine to add some code generating it to the build script; it's a simple Hello World print after all?

Uhm, if I remember correctly that testsuite is also in BinaryBuilder tests, you'd need to duplicate the generation. For quick testing is probably fine, I think it'd be nicer to eventually have the Makefile in the testsuite though.